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METHOD AND SYSTEM FOR ENHANCING AN IMAGE 
Background 

This invention relates generally to the field of signal processing and in particular 
5 to the application driven enhancement of a signal by scaling and adjustment of the 
wavelet transform coefficients associated with that signal. 

A computer system typically incorporates a monitor for the display of a signal 
representative of an image. In such a system, the image displayed on the monitor is 
frequently a digital representation of an analog image. To obtain such an image, the 

10 computer system divides the analog image into a large number of picture elements or 
"pixels," each of which corresponds to a small portion of the original image. This causes 
a loss of resolution in the displayed image. It is beneficial, therefore, to enhance the 
displayed image so as to render the digital representation of the image more faithful to 
the original analog image. 

15 For signals representative of a video image, these pixels are treated as forming a 

rectangular array consisting of rows and columns. The location of a pixel within this 
rectangular array corresponds to the location of the portion of the original image 
associated with that pixel. Thus, by referring to the row and column it occupies within 
the rectangular array, one can uniquely identify a pixel and determine its location within 

20 the original image, 

A result of arranging pixels into a rectangular array is that each pixel has 
adjacent neighbors. A pixel in the interior of the array has four adjacent neighbors. 
Except for those four pixels marking the comers of the array, a pixel at an edge has 
three adjacent neighbors. The four pixels marking the comers of the array each have two 

25 adjacent neighbors. 

In a digital representation of an image, each pixel is assigned a numerical value. 
For a monochrome image, this value typically corresponds to the brightness of the 
portion of the image corresponding to that pixel. For a color image, this value typically 
corresponds to the amoimt of red, green or blue in the portion of the image 

30 corresponding to that pixel. 

The arrangement of pixels into an array and the assignment of values to each 
pixel as described above enables a computer system for performing image enhancement 
to treat the rectangular array of pixels as samples^^/, y^) of a two-dimensional function 
y(x, y) which maps from a domain consisting of the set of all points in the image (the 

35 spatial domain) to a range consisting of the set of all brightness values in the image. 
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Each object within an image can be divided into edge pixels which correspond to 
the edges of objects with the image and interior pixels which correspond to the interiors 
of objects within the image. The edge pixels define the overall shape of the object 
whereas the interior pixels combine to show the texture of the object. Where the shape 

5 and location of an object are of interest, a computer system for image enhancement can 
amplify the edge pixels or attenuate the interior pixels, thereby rendering the edges more 
conspicuous. A prerequisite for a successful image enhancing system of this type is that 
the computer system be capable of identifying an edge. 

A pixel can generally be classified as an edge pixel by observing the difference 

10 between the value associated with that pixel and the values associated with its neighbors. 
For example, if each pixel in a row has a high value and each pixel in an adjacent row 
has a low value, one can infer that the row containing that pixel forms a horizontal edge 
and that therefore every pixel in that row is an edge pixel. Conversely, if a pixel has a 
value similar to those of all its adjacent neighbors, one can infer that that pixel is an 

15 interior pixel. 

Since the rectangxilar array of pixels can be considered samples/jcy, yD of a two- 
dimensional function^x, y\ the concept of searching for sudden changes in the pixel 
values can be likened to the process of searching for local maxima of the magnitude of 
the gradient vector V/ (x,y) associated with that function. The direction of the edge can 

20 then be associated with the direction of the gradient vector V/ (x,y) . An image 

enhancement system that identifies edge pixels by using the gradient vector in this way 
is said to operate in the spatial domain. 

Having located an edge, typical spatial domain image enhancement systems 
enhance that edge by, for example, rendering it in a different color or multiplying each 

25 pixel at an edge by a constant. This results in an image having an unnatural appearance 
resulting from enhancing only the edges of an image and leaving the remainder of the 
image untouched. 

The image function^^, y) can also be represented by a weighted sum of complex 
exponentials. This altemate representation of the image, referred to as a fi-equency 

30 domain representation, can be obtained by performing a fast Fourier transform (FFT) on 
the samplesy(x/, y^) of the image function. In this representation, the exponent of the 
complex exponential corresponds to the spatial firequency of the image and the 
weighting coefficients, which can be complex numbers, correspond to the amount of that 
spatial firequency in the image. The set of coefficients forms the firequency spectrum of 

35 the image. 
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Since high frequency coefficients generally correspond to discontinuities or 
edges in an image, an image enhancement system employing the FFT can detect edges in 
that unage by identifying high frequency components in the frequency spectrum of that 
image. 

5 Although the frequency spectrum obtained by performing an FFT on the original 

image enables an image enhancement system to detect the existence of an edge, it does 
not enable the system to determine where on the image the edge is located. This is 
because the Fourier transform provides the system only with the magnitude and phase 
corresponding to each frequency component of the original image. Consequently, the 
10 options available in such a system for enhancing edges in an image are limited. Merely 
boosting the high frequency components of the transform and then evaluating the inverse 
transform of the resuh often yields an image laced with stripes or bands at unpredictable 
locations. 

The wavelet transform, like the Fourier transform, provides a technique for 

15 decomposing a signal into a weighted sum of orthogonal basis functions. However, 
unlike the Fourier transform which uses complex exponentials as basis functions, the 
wavelet transform uses a set of basis functions, referred to as "wavelets," which can be 
non-zero only for a finite interval. A set of basis functions having tliis property is said to 
have "compact support." The general theory of wavelet transforms is set forth in 

20 Daubechies, "The Wavelet Transform, Time-Frequency Localization and Signal 

Analysis," IEEE Transactions on Information Theory, vol, 36, no. 5, Sept. 1990 which is 
hereby incorporated by reference. A computationally efficient process for carrying out 
both the wavelet transform and its inverse, is described in Mac A. Cody, "The Fast 
Wavelet Transform," Dr. Dobb 's Journal, April 1992, pp. 16-28 which is also hereby 

25 incorporated by reference. 

As is the case with Fourier transforms, a local maximum of the wavelet 
transform coefficients of an image corresponds to the presence of a high frequency 
component in the image. Unlike the Fourier transform, however, the spatial location of 
that high frequency component within the image, if basis functions having compact 

30 support are used in the wavelet transform, corresponds to the location of the high 

frequency coefficient within the array of wavelet transform coefficients. Thus, unlike the 
Fourier transform, the wavelet transform coefficients can carry information about both 
the presence of a high frequency component in the image and the location of that 
component within the image. In this way, the wavelet transform can perform edge 

35 detection. 
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Although the wavelet transform can readily locate an edge, the problem of 
rendering that edge conspicuous remains. Merely perturbing the wavelet transform of an 
image by boosting the local maxima of the wavelet transform coefficients can result in 
rendering the inverse wavelet transform so different from the original image as to be 
5 useless. Accordingly, there still exists a need in the art for employing wavelet 
transforms to better define edge regions in an image. 

Summary of the Invention 

What is necessary and desirable is a method and system for boosting the local 
10 maxima of the largest wavelet transform coefficients and then adjusting the remaining 
wavelet transform coefficients in such a way as to preserve the integrity of the inverse 
wavelet transform of the result. 

The invention encompasses a signal enhancement system and method for 
enhancing a signal represented by its wavelet transform coefficients. This is 
15 accomplished by assigning alternative values to those wavelet transform coefficients 
sharing a selected property and then assigning corresponding adjustment values to those 
coefficients lacking the selected property. These adjustment values are chosen such that 
the resulting coefficients are wavelet transform coefficients of an enhanced version of 
the original signal. 

20 According to one aspect of the invention, altemative values are optionally 

assigned to the local maxima of the wavelet transform coefficients by multiplying those 
values by a scaling constant. This choice of a selected property tends to emphasize high 
frequency components in a corresponding section of the signal. However, the choice of a 
selected property and the manner in which an altemative value is assigned can be 

25 changed depending on what features of the signal are to be enhanced. 

Similarly, the method by which the adjustment value is assigned to those 
coefficients lacking in the selected property can be adjusted to the particular application. 
For example, the adjustment value is optionally arrived at by minimizing a particular 
error. However, other types of error can be minimized without departing from the scope 

30 of the invention. 

These and other features, aspects, and advantages of the invention wall be better 
understood with reference to the following description and the accompanying drawings 
in which: 
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Description of the Drawings 

FIG. 1 depicts the overall architecture of a computer system implementing the signal 
enhancement system of the invention; 

5 FIG. 2 is a block diagram showing the signal enhancement system of the invention 
connected to a multistage wavelet transform filter; 

FIG. 3 depicts the internal architecture of the signal enhancement system depicted in 
FIG. 2; 

10 

FIG. 4 shows the fi-equency responses of the high pass filters associated with the first 
two stages of the multi-stage wavelet transform filter depicted in FIG. 2; 

FIG. 5 shows, in schematic form, the result of digitizing an image; and 

15 

FIG. 6 shows two signal enhancement systems as depicted in FIG. 2 connected in series. 

Description of the Illustrated Embodiment 

FIG. 1 depicts a signal enhancement system 60 consisting of a central processing 

20 unit ("CPU") 66 which includes a processor for executing progranuned instructions. The 
CPU 66 is in commimication with a memory unit 69. The memory imit 69 can be a 
volatile memory, such as RAM or it can be a non-volatile memory for long term storage 
of information. The CPU 66 is also in commimication with: an analog to digital 
converter 65 connected to a signal source 68, such as a video camera, a microphone, a 

25 scanner, a photocopy machine or a facsimile machine; an output device 62, such as a 
monitor, speaker, or printer; and an input device, such as a keyboard, 64 for 
communicating instructions fi-om a human operator to the CPU 66. 

The memory xmit 69 contains within it a digital representation of programmed 
instructions 67 for carrying out the signal enhancement method of the invention on a 

30 signal supplied by the signal source 68 and digitized by the A/D converter 65. The 
method to be carried out by these programmed instructions is best understood with 
reference to FIG. 2 which shows the signal enhancement system of the invention 
operating on the output of a multi-stage wavelet transform filter. 

The invention begins with the creation of a digitized representation 11 of a signal 

35 hereafter referred to as the "original signal." This can be supplied to the CPU 66 by 
passing the output 73 of an analog image source 68 through an analog to digital 
converter 65. FIG. 5 shows an analog image 76 corresponding to the analog output 
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signal 73 of an analog signal source 68. The analog output signal 68 is digitized by an 
A/D converter 65 to generate an original signal 11 representative of a video image . The 
original signal 11 can be representetd by a rectangular array 11a of picture elements, or 
"pixels," arranged in rows and colunms. Each pixel 74 is identifiable by its row 74R and 
5 its column 74C. Additionally, each pixel can have adjacent pixels 74N, 74S occupying 
the same colimm and adjacent pixels 74E, 74W occupying the same row. 

Referring again to FIG. 2, the original signal 11 is passed through a first low-pass 
filter 42a and through a first high pass filter 41a. Together, the first low-pass filter 42a 
and the first high-pass filter 41a comprise the first wavelet transform (/ = 1) 50a of a first 
10 multistage wavelet transform filter 50. The pass bands associated with these filters are 
described below. 

The output of the first low pass filter 42a is then passed through a second low 
pass filter 42b and through a second high pass filter 41b which together form the second 
wavelet transform (/ = 2) 50b of the first multistage wavelet transform fiher 50. The 
15 output of this second wavelet transform 50b can then be passed to succeeding wavelet 
transform filters 50c, 50d in the manner depicted in FIG. 2. 

Meanwhile, the outputs of the high pass filters 41a, 41b, 41c, 41d are passed 
directly to the output of the first multistage wavelet transform filter 50. The overall 
output of the first muhistage wavelet transform filter 50 thus includes the low frequency 
20 wavelet transform coefficients 13L firom the final low pass filter 42d and as many sets of 
high fi-equency wavelet transform coefficients 13H as there are high pass filters, for 
example, four sets corrsponding to the four high pass filters shown in FIG. 2, 41a, 41b, 
41c, 41d. 

In the preferred embodiment, the multistage wavelet transform filter 50 contains 
25 four stages. The filter coefficients for each of the four low pass filters in the multistage 
wavelet transform filter 50 are given by 

Hi =0.125 H3 = 0.375 

H2 = 0.375 H4 = 0.125 

The filter coefficients for the four high pass filters are as follows: 

Stages Stage 4 

-2/X3 -2IX^ 
2/X3 2IX^ 

>.3=1.03 
X4-I.OI 



Stage 1 Stage2 
'IIX^ -2/>^ 

where 

35 ^, = 1.50 

^2=1. 12 
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At each stage, the cut-off frequency for the low-pass filter and the cut-off 
frequency for the high-pass filter are chosen to divide the available frequency band in 
half as shown in FIG. 4. Thus, at the first stage 50a, the available firequency band ranges 
from 0 to yifj^y^ where /^^ is the Nyquist frequency (equal to twice the bandwidth of 

the signal). Accordingly, the first high-pass filter 41a has a filter response 72 which 
passes frequencies fi-om 1/4 /^y^ to 1/2 /^^^ and the first low-pass filter 42a has a filter 
response (not shown) which passes frequencies from 0 to 1/4 /yv^ . In the second wavelet 
transform 50b, the available frequencies extend from 0 to 1/4 /y^^ , hence the second 
high-pass filter 41b has a filter response 74 which passes frequencies from Ij^fsyq to 
V4 fNyq the low-pass filter 42b passes frequencies from 0 to 1/8 f^y^ . This process of 
subdividing the original spread of available frequencies into sub-bands one octave wide 
continues with each wavelet transform in the multi-stage wavelet transform filter 50 
illustrated in FIG. 2. 

At the last stage, the lov^ frequency wavelet transform coefficients 13L generated 
by the last low pass filter 42d are bundled with the four sets of high frequency wavelet 
transform coefficients 13H generated by the four high pass filters 41a-41d to form the 
output wavelet transform coefficients 13 of the multistage wavelet transform filter 50. 
The output wavelet transform coefficients 13 are then passed into an enhancement 
system 10 which assigns: alternative values to those high frequency wavelet transform 
coefficients 13H having a selected property; and adjusted values to the remaining high 
frequency wavelet transform coefficients 13H and low frequency wavelet transform 
coefficients 13L of the original signal. The manner in which the enhancement system 10 
assigns alternative values and adjusted values are described below with reference to 
FIG, 3. 

The enhancement system 10 generates an enhanced signal 39 which is then 
passed to an iteration controller 43 which can either direct the enhanced signal 39 back 
into the furst multistage wavelet transform filter 50 for further enhancement or send it to 
an output device such as the monitor 62 shown in FIG. 1 . If the iteration controller 43 
directs the enhanced signal 39 back into the multistage wavelet transform filter 50, then 
the enhanced signal is processed by the multistage wavelet transform filter 50 in the 
same marmer as the original signal 11. 
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FIG. 3 is a schematic block diagram of the enhancement system 10 of the 
invention. The enhancement system 10 inputs the high frequency wavelet transform 
coefficients 13H obtained from either the original signal 11 (as shown in FIG. 3) or from 
an enhanced signal 39 fed back into the multistage wavelet transform filter 50 in the 
5 manner set forth above and illustrated in FIG. 2 to a first extractor 14. The furst extractor 
14 then extracts from these high frequency wavelet transform coefficients 13H a table 
of local maxima Wj{Xf) 15, where j refers to the stage associated with the particular high 

pass filter. 

By a local maximum, we mean a wavelet transform coefficient having a 
10 magnitude in excess of the magnitudes of its neighbors. By way of example, in the video 
signal illustrated in FIG. 5, the extractor 14 compares the wavelet transform coefficient 
associated with the illustrated pixel 74 with the wavelet transform coefficients associated 
with the two adjacent pixels on the same column 74N,74S with which it shares a 
common boundary. If the magnitude of the wavelet coefficient transform associated with 
15 the illustrated pixel 74 exceeds that of its column neighbors 74N, 74S then the extractor 
14 classifies that wavelet transform coefficient as a local maximum. Similarly, in 
another embodiment, the extractor 14 compares the illustrated pixel 74 with the wavelet 
transform coefficients associated with the two adjacent pixels on the same row 74W,74E 
with which it shares a common boimdary. If the magnitude of the wavelet coefficient 
20 transform associated with the illustrated pixel 74 exceeds that of its row neighbors 74W, 
74E then the extractor 14 classifies that wavelet transform coefficient as a local 
maximum. 

The tables of local maxima 15 are then passed to a multiplier 16 for scaling by a 
scaling constant 17 thereby generating tables of scaled local maxima 18 for input to a 

25 swapper 20. The scaling constant 17 depends on which high pass filter 41a-41d 
generated the high frequency wavelet transform coefficients being processed by the 
extractor 14. In the preferred embodiment, the high frequency wavelet transform 
coefficients generated by the first high pass filter 41a are scaled by 1.6, The high 
frequency wavelet transform coefficients generated by the second high pass filter 41b 

30 are scaled by 1 .8. The high frequency wavelet transform coefficients generated by the 
third high pass filter 41c and by the fourth high pass filter 41d are scaled by 2.0 and 2.5 
respectively. 

The swapper 20 replaces the local maxima Wj(x^) 15 from the high frequency 
wavelet transform coefficients 13H by the scaled local maxima CjWpc^ 18. The set of 
35 perturbed coefficients 21 thus created by the combination of the low frequency wavelet 
transform coefficients 13L and the output of the swapper 20 is identical to the set of 
wavelet transform coefficients 13 with the exception that each local maximum from the 
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high frequency wavelet transform coefficients 13H has been multiplied by the scaling 
constant 17 corresponding to the high pass filter 41a-41d that generated it. 

It is known in the art that in order for a set of coefficients {^^ } to 

constitute a set of valid wavelet transform coefficients, certain relationships must exist 
between the coefficients. These relationships, summarized in Cody and previously 
incorporated by reference, are as follows: For a function / (x) to be represented as a 
sum of weighted wavelet basis functions: 

k 

the must satisfy: 

S^2* =1 X^*^*.»=Ofor#?iO 

* k 

k k 

where a^, is the complex conjugate of a^^. 

It is apparent, therefore, that if {a^ .a^ are valid wavelet transform 
coefficients 13 satisfying the foregoing relationships, there is no guarantee that replacing 
15 any one of the will result in a set of coefficients which continue to satisfy the 

foregoing relationships. If, for example, were a local maximum, then the coefficient 
in the set of wavelet transform coefficients 13 would be replaced by c^aj, where Cj is 

the scaling constant for stage j of the multi-stage wavelet transform filter 50 depicted in 
FIG. 2. There would then be no guarantee that the resulting set of perturbed coefficients 
20 21, namely {a^,Cja2 ,...a„ }, would continue to satisfy the foregoing conditions. In order 

to ensure that the perturbed coefficients 21 are still a valid set of wavelet transform 
coefficients, the wavelet transform coefficients 13 that are not local maxima ^<x,.) 15 
must be adjusted to correct for the perturbation caused by replacing the local maxima 
with scaled local maxima CjWfx^ 18. This adjustment preferably avoids the loss of the 
25 signal enhancement associated with having multiplied each local maximum by the 
scaling constant 17. 

The adjustment process begins by passing the perturbed coefficients 21 to a first 
multistage inverse wavelet transform filter 22 which performs the inverse of each 
wavelet transform stage 50a-50d m the first multistage wavelet transform filter 50. In 
30 the preferred embodiment, the low pass filter coefficients for all foui* low pass filters 
used in the multistage inverse wavelet transform filter 22 are as follows: 

Hi = 0.125 H3 = 0.375 
H2 = 0.375 H4 = 0.125 



wo 99/28865 



PCT/US98/25590 



- 10- 

The high pass filter coefficients for the four high pass filters used in the 
multistage inverse wavelet transform filter are as follows: 



Staeel 




Staee2 


Staee3 




Staee4 




0.0078125 




0.0078125X2 


0.0078125 


■h 


0.0078125 


X4 


0.0546850 


h 


0.0546850X2 


0.0546850 


■h 


0.0546850 


X4 


0.1718750 


h 


0.1718750X2 


0.1718750 


■h 


0.1718750 


\ 


-0.1718750 


h 


0.1718750X2 


-0.1718750 


■h 


-0.1718750 


X4 


-0.0546850 




0.0546850-X2 


-0.0546850 


■h 


-0.0546850 


X4 


-0.0078125 


h 


-0.0078 125-X2 


-0.0078125 


■h 


-0.0078125 


X4 
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where 

>.i = 1.50 ^3 = 1.03 
>.2= 1.12 ?.4=1.01 

15 The output of the first multistage inverse wavelet transform filter 22 is then 

passed to a second multistage wavelet transform filter 24 identical to the multistage 
wavelet transform filter 50 shown in FIG. 2. The second multistage wavelet transform 
filter 24 performs the same sequence of wavelet transforms performed by the first 
multistage wavelet transform filter 50 as shown in FIG. 2. This resuhs in a set of 

20 estimated coefficients 25. 

It is important to note that despite the manner in which they were generated, the 
estimated coefficients 25 are not necessarily identical to the perturbed coefficients 21 
firom which they were derived. This is because the mapping provided by the wavelet 
transform fi-om the spatial domain to the frequency domain is not one-to-one and onto. 

25 The mathematical theory of this aspect of the wavelet transform is discussed in Jian Lu, 
Signal Recovery and Noise Reduction with Wavelets^ Ph.D. Thesis, June 1993, 
Dartmouth College, Hanover, N.H., which is hereby incorporated by reference. 

A second extractor 26, identical to the first extractor 14, extracts from the 
estimated coefficients 25 those coefficients located at the rows and columns 

30 corresponding to the original local maxima H^(jc,.) 15, The output of the second extractor 
26 is a table of estimates 27 Wj (x^ ) having as many entries as there were in the table of 

local maxima 15. A first summer 30 substracts the entries from the table of estimates 27 
from the corresponding elements of the table of local maxima 15 thereby generating an 
error table 31, each element of which corresponds to a sampled value of an error 
35 fimction 8y(x/) evaluated at the locations corresponding to the locations of the local 
maxima ac/. 
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The error table 31 corresponding to sampled values of the error function is then 
passed to an evaluator 32 which derives a function e^. (x). This function is chosen such 

that when added to the perturbed coefficients 21, the resulting output set of wavelet 
transform coefficients 37 satisfies two conditions: (1) at the rows and columns 
corresponding to the local maxima Wjix^) IS of the wavelet transform coefficients of the 
original image, the output set of wavelet transfonn coefficients 37 contains the 
corresponding local maxima 15; and (2) the sum of the differences between the 
coefficients of the output set of wavelet transform coefficients 37 and the corresponding 
coefficients firom the wavelet transform coefficients 13 passed to the enhancement 
system 10 by the multistage wavelet transform filter 50 and the rate of change of the 
differences between the coefficients of the output set of wavelet transform coefficients 
37 and the corresponding coefficients from the wavelet transform coefficients 13 is 
minimized. The method used by the evaluator 32 to derive such a function Zj(x) is set 
forth below. 

Because of condition (1), we require that at those locations associated with the 
local maxima 15 of the original signal, the error function be constrained to be equal to 
the difference between the scaled local maximum Cj Wjix^) 18 of the original signal and 

the value of the estimated coefficient 27 at the corresponding location. In other words: 
Sj (Xf ) = CjWj(Xi )-Wj {Xf ) for / = 1 to njnaximaj 

where xy is the local maximum and njnaximaj is the number of local maxima at stage 

7- 

Condition (2) can be satisfied by minimizing, for each gap between consecutive 
local maxima X,- and x,+], the definite integral: 

where the second term of the integrand is included to prevent spurious local maxima 
from distorting the solution. 

The above definite integral can be minimized by solving the differential equation 

dx 

the general solution of which is: 

ey(x) = aexp(2 ■'x) + Pexp(-2 ^jc) 



dx 



\dx 
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The constants a and p are then chosen to satisfy the boundary conditions 
imposed by condition (1) at and x^+j . The resulting a and P are given by 

^ _ S/(x,)exp(-2-^x,,0-Sy(x,,i)exp(-2->3Cy) 
exp (2-^ X,. - - exp (2"^x,^i - 2"^ x,) 

and 

P = (s/x,)-aexp(2-^x,))exp(2-^x,0 
The output of the error function evaluator 32 is an error function z(x) 33 which 
can be added to the perturbed coefficients 21 at a second summer 34. The result of this 
addition is a set of output wavelet transform coefficients 37 satisfying the two conditions 
set forth above and corresponding to an enhanced signal 39. This set of output wavelet 
transform coefficients 37 is then passed to a second multistage inverse wavelet transform 
filter 36 identical to the first multistage transform filter 22. The second multistage 
inverse wavelet transform filter 36 performs the inverse of each wavelet transform stage 
50a-50d performed by the first multistage wavelet transform filter 50 thereby 
generating the enhanced signal 39. 

Referring now to FIG. 2, the resulting enhanced signal 39 corresponding to the 
original image 11 can then be directed by the iteration controller 43 to an output device 
such as the monitor 62 shown in FIG. 1 . Alternatively, the resulting enhanced signal 39 
can be fed back, by the iteration controller 43, into the first multistage wavelet transform 
filter as shown in FIG. 2 for further enhancement and the process repeated iteratively 
either a predefined number of times or until the error has been reduced below a 
predefined level. 

The foregoing system can be used to enhance, in a variety of different ways, a 
one dimensional signal, such as an audio signal, or an Az-dimensional signal. 

Different enhancement results can be achieved by emphasizing specific 
firequency bands relative to the entire signal bandwidth. For example, instead of 
extracting a table of local maxima, the first extractor 14 can extract wavelet transform 
coefficients having other selected properties. The first extractor 14 can, for example, 
extract a table of those coefficients exceeding a specified value or a table of those 
coefficients falling within a range of values. 

The assignment of an altemative value to the coefficients classified by the first 
extractor 14 as having k selected property can likewise take on many forms. For 
example, instead of multiplying the coefficient by a constant 17 to obtain an altemative 
value 18, as depicted in FIG. 3, one could obtain an altemative value 18 by adding an 
offset or by convolving the selected coefficient with a predefined sequence. 
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For those coefficients lacking the selected property, the step of assigning an 
adjustment value can also be altered without departing from the scope of the invention. 
For example, a different integrand can be used or different boundary conditions can be 
imposed by the evaluator 32. 
5 When processing a signal representative of an image as shown in FIG. 5, the first 

extractor 14 determines whether a particular wavelet transform coefiBcient 74 is a local 
maximum by comparing its value by that of its adjacent neighbors, either in the same 
row 74R or in the same colunm 74C, but preferably not both. Further enhancement of 
the image can, however, be achieved by comparing the value of the wavelet transform 

10 coefficient 74 with values of wavelet transform coefficients in both the same row 74R 
and in the same column 74C. This can be implemented by passing the output of the first 
enhancement system 10 to a second multistage wavelet transform filter 52 as shown in 
FIG. 6. The output of the second wavelet transform filter 52 is then passed to a second 
enhancement system 12 similar to that shown in FIG. 2 with the exception that the first 

15 extractor 14 obtains local maxima by comparing each pixel with those adjacent pixels 
not used by the first extractor of the first enhancement system 10. Thus, if the first 
enhancement system 10 were to obtain local maxima by comparing each pixel 74 with 
adjacent pixels on the same row 74E, 74W, the second enhancement system 12 would 
obtain local maxima by comparing each pixel 74 with those adjacent pixels in the same 

20 colunm 74N, 74S. Conversely, if the first enhancement system 10 were to obtain local 
maxima by comparing each pixel 74 with adjacent pixels on the same column 74N, 74S, 
, the second enhancement system 12 would obtain local maxima by comparing each pixel 
74 with those adjacent pixels in the same row 74E, 74W. 

The wavelet transform coefficients generated by the second multistage wavelet 

25 transform filter 52 are passed to a second enhancement system 12 identical to the first 
enhancement system 10 with the exception that the extractor associated with the second 
enhancement system 12 determines whether a particular wavelet transform coefficient 74 
is a local maximimi by comparing it with its neighbors in the direction orthogonal to that 
used by the extractor in the first enhancement system 10. For example, if the first 

30 enhancement system 10 determines whether a wavelet transform coefficient 74 is a local 
maximum by comparing it v^th its neighbors 74E, 74W in the same row 74R, the 
second enhancement system 12 will determine whether a particular wavelet transform 
coefficient 74 is a local maximum by comparing it with its neighbors 74N, 74S in the 
same colunm 74C. Conversely, if the first enhancement system 10 determines whether a 

35 particular wavelet transform coefficient 74 is a local maximum by comparing it with its 
neighbors 74N, 74S in the same colimm 74C, then the second enhancement system 12 
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will determine whether a particular wavelet transform coefficient 74 is a local maximum 
by comparing it with its neighbors 74E, 74W in the same row 74R, 

It is thus seen that the invention efficiently attains the objectives set forth above, 
among those made apparent fi-om the preceding description. Since certain changes may 
be made in the above constructions without departing from the scope of the invention, it 
is intended that all matter contained in the above description or shown in the 
accompanying drawings be interpreted as illustrative and not in a limiting sense. 

It is also to be understood that the following claims are to cover all generic and 
specific features of the invention described herein, and all statements of the scope of the 
invention which, as a matter of language, might be said to fall therebetween. 

Having described the invention and a preferred embodiment thereof, what is 
claimed as new and secured by Letter Patent is: 
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We Claim; 

1 . A method for generating, on a computer system, an enhanced signal from 
an original signal, said original signal being represented by a first set of wavelet 

5 transform coefficients including at least one coefficient having a selected property and at 
least one coefficient lacking said selected property and said enhanced signal being 
represented by a second set of wavelet transform coefficients, said method comprising 
the steps of: 

replacing said at least one coefficient having a selected property with an 
10 altemative value, thereby forming a set of alternative coefficients; and 

replacing said at least one coefficient lacking said selected property with 
an adjusted value, thereby forming a set of adjusted coefficients, said adjusted value 
being chosen such that said set of adjusted coefficients and said set of altemative 
coefficients define a second set of wavelet transform coefficients representative of said 
15 enhanced signal. 

2. The method of claim 1 , further comprising the step of selectmg said 
original signal to be a video signal, an audio signal, or an audiovisual signal, and 
optionally wherein said altemative value is the product of a predefined scaling constant 

20 and said at least one coefficient having said selected property. 



3. The method of claim 1 or claim 2, wherein said at least one coefficient 
having said selected property acquires said selected property independently of the values 
of the remaining coefficients from said first set of wavelet transform coefficients. 

25 

4. The method of any of the preceding claims, wherein said selected 
property is the property of being a local maximum of said first set of wavelet transform 
coefficients and wherein said step of replacing said at least one coefficient having said 
selected property further comprises the step of determining whether a candidate 

30 coefficient from said first set of wavelet transform coefficients is a local maximum. 



5. The method of any of the preceding claims, wherein said selected 
property is the property of having a value such that the difference between said value 
and a value of a different wavelet transform coefficient from said first set of wavelet 
35 transform coefficients exceeds a threshold. 
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6. The method of any of the preceding claims, wherein said step of replacing 
said at least one coefficient having a selected property includes the step of determining 
whether a candidate coefficient has said selected property by comparing the value of said 
candidate coefficient with the value of at least one neighboring coefficient from said first 
set of wavelet transform coefficients, 

7. The method of any of the preceding claims, wherein said first set of 
wavelet transform coefficients comprises a neighboring coefficient adjacent to said 
candidate coefficient, and/or optionally 

wherein said adjusted value is further chosen to minimize an average 
difference between said alternative coefficients and said set of wavelet transform 
coefficients lacking said selected property. 

8. The method of any of the preceding claims, further comprising the step of 
selecting said adjusted value to minimize an average of a sum of a first term comprising 
a difference between said alternative coefficients and said set of wavelet transform 
coefficients lacking said selected property and a second term comprising the rate of 
change of said difference. 

9. The method of any of the preceding claims, wherein said step of replacing 
said at least one coefficient lacking said selected property comprises the step of 

applying to said alternative coefficients the inverse of the wavelet 
transform used to obtain said first set of wavelet transform coefficients from said signal, 
thereby generating a set of inverse wavelet transform coefficients, and optionally the 
step of 

generating a set of estimated coefficients by applying to said inverse 
wavelet transform coefficients a wavelet transform identical to the wavelet transform 
used to obtain said first set of wavelet transform coefficients, and optionally wherein 
said evaluating step further comprises the steps of 

indexing said set of estimated coefficients and said set of alternative 

coefficients, 

determining an index corresponding to said at least one coefficient having 
said selected property, and 

extracting, from said set of estimated coefficients, a corresponding 
estimated coefficient associated with said index. 
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10. The method of any of the preceding claims, wherein said step of replacing 
said at least one coefficient lacking said selected property with an adjusted value 
comprises the step of selecting a correction function such that said second set of wavelet 
transform coefficients: 

5 includes said alternative coefficient at a location corresponding to the 

location of said at least one coefficient having said selected property; and 

minimizes an error between said second set of wavelet transform 
coefficients and said first set of wavelet transform coefficients evaluated at at least one 
location other than said location corresponding to said at least one coefficient having 
10 said selected property, and optionally comprising the step of 

selecting said correction function to minimize the rate of change of said 
error between said second set of wavelet transform coefficients and said first set of 
wavelet transform coefficients evaluated at at least one location other than said location 
corresponding to the location of said at least one wavelet transform coefficient having 
15 said selected property in said first set of wavelet transform coefficients. 

1 1 . The emthod of claim 1 , wherein said step of replacing said at least one 
coefficient having a selected property comprises the steps of 

generating a set of estimated coefficients by evaluating a wavelet 
20 transform of an inverse wavelet transform of said set of alternative coefficients, said set 
of estimated coefficients having an element corresponding to said at least one coefficient 
having said selected property, 

evaluating the difference between said at least one coefficient having said 
selected property and said element corresponding to said at least one coefficient having 
25 said selected property, and 

deriving a set of values of a correction function which when added to said 
set of alternative coefficients results in said set of wavelet transform coefficients 
representative of said enhanced version of said signal, and optionally 

wherein said alternative value is the product of a predefined scaling 
30 constant and said at least one coefficient having said selected property. 
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1 2. The method of claim 1 1 , further comprising the step of choosing said 
selected property to be the property of being a local maximum of said first set of wavelet 
transform coefficients and wherein said step of replacing said at least one coefficient 
having said selected property includes the step of determining whether a candidate 
5 coefficient firom said first set of wavelet transform coefficients is a local maximum, and 
optionally wherein said determining step includes the step of comparing the value of 
said candidate coefficient with that of a neighboring coefficient fi-om said first set of 
wavelet transform coefficients. 

10 13. The method of claim 1 1 or claim 12, wherein said at least one coefficient 

having said selected property acquires said property independently of the values of the 
remaining coefficients from said first set of wavelet transform coefficients. 

14. The method of claims 11 to 13, further comprising the step of choosing 
1 5 said selected property to be the property of having a value between a first threshold 

value and a second threshold value, and optionally 

wherein said selected property is the property of having a value such that 
the difference between said value and a value of a different wavelet transform coefficient 
from said first set of wavelet transform coefficients exceeds a threshold value. 

20 

15. The method of claims 1 1 to 14, wherein said evaluating step further 
comprises the steps of 

indexing said set of estimated coefficients and said set of alternative 

coefficients, 

25 determining an index corresponding to said at least one coefficient having 

said selected property, and 

extracting, from said set of estimated coefficients, an estimated 
coefficient corresponding to said index, and optionally the step of 

subtracting said at least one coefficient having said selected property 
30 firom said corresponding estimated coefficient. 
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16. The method of claims 1 1 to 15, wherein said deriving step further 
comprises the step of selecting said correction function such that said second set of 
wavelet transform coefficients, 

includes said alternative coefficient at a location corresponding to the 
location of said at least one coefficient having said selected property, and 

minimizes an error between said second set of wavelet transform 
coefficients and said first set of wavelet transform coefficients evaluated at at least one 
location other than said location corresponding to said at least one coefficient having 
said selected property, and optionally the step of 

selecting said correction function to minimize the rate of change of said 
error between said second set of wavelet transform coefficients and said first set of 
wavelet transform coefficients evaluated at at least one location other than said location 
corresponding to the location of said at least one wavelet transform coefficient having 
said selected property in said fu-st set of wavelet transform coefficients. 

17. The method of claim 16, wherein said deriving step further comprises the 
step of selecting said correction function to be a solution to a membrane equation. 

18. The method of claim 1 1, wherein said step of replacing said at least one 
coefficient having a selected property comprises the step of replacing said at least one 
local maximum by a scaled local maximum thereby forming a perturbed set of wavelet 
transform coefficients, and optionally 

said step of generating a set of estimated coefficients by evaluating a 
wavelet transform of an inverse wavelet transform of said set of altemative coefficients 
comprises the step of generating a set of estmiated coefficients by evaluating a wavelet 
transform of an inverse wavelet transform of said perturbed set of wavelet transform 
coefficients, said set of estimated coefficients having an element corresponding to said 
scaled local maximum, and optionally 

said step of evaluating the difference comprises the step of evaluating the 
difference between said scaled local maximum and said element corresponding to said 
scaled local maximum, and optionally 

said step of deriving said set of values of said correction function 
comprises the step of adding said values of said correction function to said set of 
perturbed wavelet transform coefficients to generate said set of wavelet transform 
coefficients representative of said enhanced version of said signal. 
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19. The method of claim 18, wherein said first set of wavelet transform 
coefficients comprises rows and columns of an image and said neighboring coefBcient is 
on the same row as said candidate coefficient, and optionally wherein said neighboring 
coefficient is adjacent to said candidate coefficient. 

5 

20. A system for generating an enhanced signal from an original signal, said 
original signal being represented by a first set of wavelet transform coefficients 
including at least one coefficient having a selected property and at least one coefficient 
lacking said selected property and said enhanced signal being represented by a second 

10 set of wavelet transform coefficients, said system comprising: 

first replacing means for replacing said at least one coefficient having a 
selected property with an alternative value, thereby forming a set of alternative 
coefficients, and 

second replacing means for replacing said at least one coefficient lacking 
15 said selected property with an adjusted value, thereby forming a set of adjusted 

coefficients, said adjusted value being chosen such that said set of adjusted coefficients 
and said set of alternative coefficients define a second set of wavelet transform 
coefficients representative of said enhanced signal. 

20 21. The system of claim 20, wherein said alternative value is the product of a 

predefined scaling constant and said at least one coefficient having said selected 
property, and/or 

wherein said at least one coefficient having said selected property 
acquires said selected property independently of the values of the remaining coefficients 

25 fi-om said first set of wavelet transform coefficients, and/or 

wherein said selected property is the property of being a local maximum 
of said first set of wavelet transform coefficients and wherein said first replacing means 
further comprises means for determining whether a candidate coefficient fix)m said first 
set of wavelet transform coefficients is a local maximum, and/or 

30 wherein said selected property is the property of having a value such that 

the difference between said value and a value of a different wavelet transform coefficient 
fi-om said first set of wavelet transform coefficients exceeds a threshold. 



22. The system of claim 20 or claim 21 , wherein said first replacing means 
35 comprises means for determining whether a candidate coefBcient has said selected 
property by comparing the value of said candidate coefficient with the value of at least 
one neighboring coefficient. 
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23. The system of claims 20 to 22, further comprising means for selecting 
said adjusted value to minimize an average difference between said alternative 
coefficients and said set of v^ravelet transform coefficients lacking said selected property, 
and/or further comprising 

means for selecting said adjusted value to minimize an average of a simi 
of a first term comprising a difference between said alternative coefficients and said set 
of wavelet transform coefficients lacking said selected property and a second term 
comprising the rate of change of said difference. 

24. The system of claims 20 to 23, wherein said second replacing means 
comprises means for applying to said ahemative coefficients the inverse of the wavelet 
transform used to obtain said first set of wavelet transform coefficients from said signal, 
thereby generating a set of inverse wavelet transform coefficients, and optionally 
comprises 

means for generating a set of estimated coefficients by applying to said 
inverse wavelet transform coefficients a wavelet transform identical to the wavelet 
transform used to obtain said first set of wavelet transform coefficients. 

25. The system of claims 20 to 24, wherein said second replacing means 
further comprises 

means for indexing said set of estimated coefficients and said set of 
alternative coefficients, 

means for determining an index corresponding to said at least one 
coefficient having said selected property, and 

means for extracting, based on said index, the value of a corresponding 
estimated coefficient fi-om said set of estimated coefficients. 
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26. The system of claims 20 to 25, wherein said second replacing means 
comprises means for selecting a correction function such that said second set of wavelet 
transform coefficients: 

includes said alternative coefficient at a location corresponding to the 
5 location of said at least one coefficient having said selected property, and 

minimizes an error between said second set of wavelet transform 
coefficients and said first set of wavelet transform coefficients evaluated at at least one 
location other than said location corresponding to said at least one coefficient having 
said selected property, and optionally comprises 
10 means for selecting said correction function to minimize the rate of 

change of said error between said second set of wavelet transform coefficients and said 
first set of wavelet transform coefficients evaluated at at least one location other than 
said location corresponding to the location of said at least one wavelet transform 
coefficient having said selected property in said first set of wavelet transform 
15 coefficients. 

27. The system of claim 20, wherein said second replacing means comprises 
means for generating a set of estimated coefficients by evaluating a 

wavelet transform of an inverse wavelet transform of said set of alternative coefficients, 
20 said set of estimated coefficients having an element corresponding to said at least one 

coefficient having said selected property, 

means for evaluating the difference between said at least one coefficient 

having said selected property and said element corresponding to said at least one 

coefficient having said selected property, and 
25 means for deriving a set of values of a correction function which when 

added to said set of alternative coefficients results in said set of wavelet transform 

coefficients representative of said enhanced version of said signal, and optionally 
wherein said alternative value is the product of a predefined scaling 

constant and said at least one coefficient having said selected property. 

30 

28. The system of claim 27, wherein said selected property is the property of 
being a local maximum of said first set of wavelet transform coefficients and wherein 
said replacing means further comprises means for determining whether a candidate 
coefficient from said first set of wavelet transform coefficients is a local maximum. 



35 
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29. The system of claim 27 or claim 28, wherein said at least one coefficient 
having said selected property acquires said property independently of the values of the 
remaining coefficients from said first set of wavelet transform coefficients. 

30. The system of claims 27 to 29, wherein said selected property is the 
property of having a value such that the difference between said value and a value of a 
different wavelet transform coefficient from said first set of wavelet transform 
coefficients exceeds a threshold value. 

3 1 . The system of claim 28, wherein said determining means mcludes means 
for comparing the value of said candidate coefficient with that of a neighboring 
coefficient from said first set of wavelet transform coefficients. 

32. The system of claims 27 to 3 1 , wherein said evaluating means further 
comprises 

means for indexing said set of estimated coefficients and said set of 
alternative coefficients, 

means for determining an index corresponding to said at least one 
coefficient having said selected property, and 

means for extracting the value of a corresponding estimated coefficient 
from said set of estimated coefficients corresponding to said index. 

33. The system of claims 27 to 32, wherein said deriving means further 
comprises means for selecting said correction function such that said second set of 
wavelet transform coefficients: 

includes said alternative coefficient at a location corresponding to the 
location of said at least one coeflBcient having said selected property, and 

minimizes an error between said second set of wavelet transform 
coefficients and said first set of wavelet transform coefficients evaluated at at least one 
location other than said location corresponding to said at least one coefficient having 
said selected property, and optionally 

wherein said deriving means further comprises means for selecting said 
correction function to minimize the rate of change of said error between said second set 
of wavelet transform coefficients and said first set of wavelet transform coefficients 
evaluated at at least one location other than said location corresponding to the location of 
said at least one wavelet transform coefficient having said selected property in said first 
set of wavelet transform coefficients. 
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34. The system of claim 27, wherein said first replacing means comprises 
third replacing means for replacing said at least one local maximum by a scaled local 
maximum thereby forming a perturbed set of wavelet transform coefiBcients, and 
optionally 

5 wherein said generating means comprises second generating means for 

generating a set of estimated coefficients by evaluating a wavelet transform of an inverse 
wavelet transform of said perturbed set of wavelet transform coefficients, said set of 
estimated coefficients having an element corresponding to said scaled local maximum, 
and optionally 

10 wherein said evaluating means comprises second evaluating means for 

evaluating the difference between said scaled local maximum and said element 

corresponding to said scaled local maximum, and optionally 

wherein said deriving means comprises second deriving means for 

deriving a set of values of a correction function which when added to said set of 
15 perturbed wavelet transform coefficients results in said set of wavelet transform 

coefficients representative of said enhanced version of said signal. 

35. The system of claim 34, further comprising 

means for indexing said set of estimated coefficients and said set of 
20 perturbed wavelet transform coefficients, 

means for determining an index corresponding to said scaled local 

maximum, and 

means for extracting the value of the corresponding element fi-om said set 
of estimated coefficients corresponding to said index. 

25 
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36. The system of claim 34, wherein said second deriving means fiirther 
comprises means for selecting said correction function such that said second set of 
wavelet transform coefficients: 

includes said scaled local maximum at a location corresponding to the 
location of said at least one local maximum in said first set of wavelet transform 
coefficients, and 

minimizes an error between said second set of wavelet transform 
coefficients and said first set of wavelet transform coefficients evaluated at at least one 
location other than said location corresponding to the location of said at least one local 
maximiim, and optionally 

wherein said second deriving means further comprises means for 
selecting said correction function to minimize the rate of change of said error between 
said second set of wavelet transform coefficients and said first set of wavelet transform 
coefficients evaluated at at least one location other than said location corresponding to 
the location of said at least one local maximum in said first set of wavelet transform 
coefficients. 
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